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Abstract — We recently proposed a new coding scheme for 
the L-channel multiple descriptions (MD) problem for general 
sources and distortion measures involving 'Combinatorial Mes- 
sage Sharing' (CMS) |7| leading to a new achievable rate- 
distortion region. Our objective in this paper is to establish that 
this coding scheme strictly subsumes the most popular region for 
this problem due to Venkataramani, Kramer and Goyal (VKG) 
(31 . In particular, we show that for a binary symmetric source 
under Hamming distortion measure, the CMS scheme provides 
a strictly larger region for all L> 2. The principle of the CMS 
coding scheme is to include a common message in every subset 
of the descriptions, unlike the VKG scheme which sends a single 
common message in all the descriptions. In essence, we show that 
allowing for a common codeword in every subset of descriptions 
provides better freedom in coordinating the messages which can 
be exploited constructively to achieve points outside the VKG 
region. 

Index Terms — Multiple descriptions coding, Source coding. 
Rate distortion theory 



I. Introduction 

The Multiple Descriptions (MD) problem has been studied 
extensively since late 1970s, see UJ, ||2J, |3|, |4|, |5|, 16J 
and the references therein. In a i— descriptions MD setup, 
the encoder sends L packets (descriptions) which are sent to 
the receiver over L different channels. In the most general 
setting, it is assumed that the decoder receives a subset of 
the descriptions without any error and the remaining are 
completely lost. The decoder reconstructs the source upto a 
given level of distortion when a subset of the descriptions 
are received. The goal of the MD problem is to establish the 
complete rate-distortion region to trade-off the encoding rates 
to the achievable distortions. The general setup has remained 
challenging and unsolved due to the intricacies of the problem 
in maintaining the balance between the full reconstruction 
quality versus quality of individual descriptions. 

Until recently, for general sources and distortion measures, 
the most recognized achievable rate-distortion region for the 
i— channel MD setup was due to Venkataramani, Kramer and 
Goyal (VKG) Q, whose encoding scheme builds on the prior 
work for the 2-channel case by El-Gamal and Cover (EC) [1| 
and Zhang and Berger (ZB) 12). The VKG scheme involves a 
combinatorial number of refinement codebooks along with a 
single shared codebook used to control the redundancy across 
the descriptions. We introduced a new encoding scheme in 



fTj^involving 'Combinatorial Message Sharing' (CMS) which 
differs from the VKG scheme primarily in the number of 
shared codebooks. The CMS scheme allows for every subset 
of the descriptions to share a different common codebook, 
thereby leading to a combinatorial number of shared messages. 
At the time of submission of |7|, it was not known whether 
the CMS scheme leads to a strictly improved rate-distortion 
region over the VKG scheme. In this paper, our objective is 
to prove by example that the new region is indeed strictly 
better Specifically, we show that for a binary symmetric source 
under Hamming distortion measure, the CMS scheme achieves 
points outside the VKG region VL > 2. In fact, more generally, 
our result holds VL > 2 for any source and distortion measures 
for which the ZB scheme achives points outside the EC 
scheme for the corresponding 2-descriptions problem. We note 
in passing that, other encoding schemes have been proposed 
in the literature for certain special cases (specific sources 
and distortion measures) of the L— channel MD setup |6|, 
which achieve points outside TZVvkg- However, none of 
these schemes have been proven to subsume or outperform 
TZVvkg for general sources and distortion measures. The 
potential implications of our results on these coding schemes 
are beyond the scope of this paper In the following Section, 
we formally state the i— channel MD setup and describe the 
prior results due to EC Q, ZB 12, VKG El and the CMS 



scheme fl\. In section III we prove the strict improvement of 
the achievable region. 

II. Formal Definitions and Prior results 

We follow the notation in ||7J. A source produces n iid 
copies, denoted by X" = {X^^\X^'^'> . . . , X^")), of a generic 
random variable X taking values in a finite alphabet X. We 
denote £ = {1,...,L}. There are L encoding functions, 
/;(•) I E C, which map X" to the descriptions J; = 
where J; e {1, . . . Bi } for some Bi > 0. The rate of descrip- 
tion I is defined as Ri = \og2{Bi). Each of the descriptions 
are sent over a separate channel and are either received at the 
decoder error free or are completely lost. There are 2^ — 1 
decoding functions for each possible received combination of 
the descriptions X^ = (^^ \ • • • ,^1;"') = 9k{Ji ■ I IC), 

V/C C C, K, ^ (j>, where Xfc takes on values on a finite set Xjc, 
and (p denotes the null set. When a subset JC of the descrip- 
tions are received at the decoder, the distortion is measured 
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Figure 1: Codebook generation for VKG coding scheme. Black 
(V^) indicates 'common random variable'. Red (Ui I G C) in- 
dicates 'base layer random variables' and White (Us \S\ > 1) 
indicates 'refinement random variables'. The arrow indicates 
the order of codebook generation. 



as Dk. = E 

distortion measures djc{-) defined as djc 



for some bounded 

X X Xk. ^ n. 



We say that a rate-distortion tuple [Ri^Djc : i G £,/C C 
£,/C 7^ 0) is achievable if there exit L encoding functions 
with rates (i?i . . . , Rl) and 2^ — 1 decoding functions yielding 
distortions Diq. The closure of the set of all achievable 
rate-distortion tuples is defined as the 'L-channel multiple 
descriptions RD region' . Note that, this region has L + 2^ - 1 
dimensions. 

In what follows, 2"^ denotes the set of all subsets (power 
set) of any set S and \S\ denotes the set cardinality. Note that 
|2'^| = 2l'^l. S"^ denotes the set complement. For two sets Si 
and 1S2, we denote the set difference hy Si — S2 ~ {IC : IC E 
5i, /C ^ 52}. We use the shorthand {U}s for {C/k; : /C G Sj^ 

A. VKG Encoding Scheme 

The achievable region of ^ is denoted here TZVvkg and 
is described as follows. Let (Vc, {U}2'-~<j}) be any set of 2^ 
random variables distributed jointly with X. Then, an RD tuple 
is said to be achievable if there exist functions 1/^5 (•) such that: 
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E[ds{X,^s{VcAU}2S-^))] 
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(2) 



yS C C. The closure of the achievable tuples over all 
such 2^ random variables gives TZVykg- Here, we only 
present an overview of the encoding scheme. The order of 
codebook generation of the auxiUary random variables is 
shown in Figure [l] First, 2"^^ codewords of Vc are gen- 
erated using the marginal distribution of Vc- Conditioned on 
each codeword of Vc, 2"^' codewords of Ui are generated 

-Note the difference between {U}s ™d Us- {U}s is a set of variables, 
whereas f/5 is a single variable. 



according to their respective conditional densities. Next, for 
each j G (1, . . . , 2"^^'^'''^is'c ^'''), a single codeword is 
generated for C//cO') conditioned on (w£(j), {u(j)}2'C-0-/c) 
V/C C £, |/C| > 1. Note that to generate the codebook for 
Uic, we first need the codebooks for all {?7}2'C-0-k; ^^d Vc- 
On observing a typical sequence x", the encoder tries to 
find a jointly typical codeword tuple one from each codebook. 
Codeword index of Ui (at rate R^) is sent in description 
I. Along with the 'private' messages, each description also 
carries a 'shared message' at rate which is the codeword 
index of Vc- Hence the rate of each description is Ri — Ri + 
R^- VKG showed that, to ensure finding a set of jointly typical 
codewords with the observed sequence, the rates must satisfy 
([TJ. It then follows from standard arguments (see for example 
"typical average lemma" fSl) that, if the random variables 
also satisfy then the distortion constraints are met. Note 
that, Vc is the only shared random variable. Ui : I E C form 
the base layer random variables and all Ujc ■ \IC\ > 2 form 
the refinement layers. Observe that the codebook generation 
follows the order: shared layer base layer refinement 
layer 

The VKG scheme for the 2-descriptions scenario involves 
4 auxiliary random variables V12, Ui,U2 and U12- The VKG 
region was originally derived as an extension of the EC HI 
and ZB [2J coding schemes, which were designed for the 2- 
descriptions scenario. The first of the two regions was by 
El-Gamal and Cover and their rate region (denoted here by 
TZVec) is obtained by setting V12 = $ in TZVvkg, where $ 
is a constant. Zhang and Berger (their region is denoted here 
by TZVzb) later showed that, including the shared random 
variable can give strict improvement over TZVec- Their result, 
while perhaps counter-intuitive at first, clarifies the fact that, 
a shared message among the descriptions helps to better 
coordinate the messages, thereby providing a strictly improved 
RD region, even though it introduces redundancy. We will 
describe their result in detail in Section [In] as our example 
builds upon theirs. However, it is known that TZVec is 
complete for some special cases of the setup (see for example 
|4|, M). 



B. CMS Encoding Scheme 

In this section, we briefly describe our CMS encoding 
scheme in Q. The VKG encoding scheme employs one 
common codeword (Vc) that is sent in all the L descriptions. 
However, when dealing with L > 2 descriptions, restricting 
to a single shared message could be suboptimal. The CMS 
scheme therefore allows for 'combinatorial message sharing', 
i.e a common codeword is sent in each (non-empty) subset of 
the descriptions. Before describing the codebook generation 
and stating the theorem, we define the following subsets of 
2^: 



1-W+ 



{S--SE 2^, \S\ = W} 
{S--S E 2^, \S\ > W} 



(3) 
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Figure 2: Codebook generation for the CMS coding scheme. 



Let B be any non-empty subset of C with \B\ < W. We define 
the following subsets of lyir and Iw+' 



{S -.S elw: BCS} 
{S :S elw+, BCS} 



(4) 



{5:5C£,|5| > I,|/Cn5| > 0} (5) 



:iw{B) 

Tw+{B) 
We also defin^ 

J{K.)^{jx,+ {l) 

The shared random variables are denoted by 'V\ The 
base and the refinement layer random variables are denoted 
by '[/'. The codebook generation is done in an order as 
shown in Figure |2] First, the codebook for Vc is generated. 
Then, the codebooks for Vg, \S\ = W are generated in the 
order W ^ L - I, L ~ 2 . . .2. 2"^'e codewords of Vq are 
independently generated conditioned on each codeword tuple 
of {^}x„,+ (S)- """hi^ followed by the generation of the base 
layer codebooks, i.e. Ui, I G C. Conditioned on each codeword 
tuple of 2"-''' codewords of Ui are generated 

independently. Then the codebooks for the refinement layers 
are formed by generating a single codeword for Ug, |5| > 1 
conditioned on every codeword tuple of {{V} j(s), {U}2ss)- 
Observe that the base and the refinement layers in the CMS 
scheme are similar to that in the VKG scheme, except that 
they are now generated conditioned on a subset of the shared 
codewords. 

The encoder employs joint typicality encoding, i.e., on 
observing a typical sequence x", it tries to find a jointly 
typical codeword tuple, one from each codebook. As with the 
VKG scheme, the codeword index of Ui (at rate i?;) is sent 
in description I. However, now the codeword index of Vs (at 
rate R^) is sent in all the descriptions I E S. Therefore the 
rate of description I is: 



Ri — Ri 



E 



R. 



(6) 



We next state the main result in Q which describes a new 
region for the L— Channel MD setup achievable by the CMS 
scheme. Let {{V}j(^c) , {C/}2^-0) be any set of 2^+^ - L - 2 

'We use the notation ^ C B to mean A- is subsumed in B and ^ C B to 
mean strictly subsumed. 



random variables jointly distributed with X. We define the 
quantities aw{Q) and P{S) as follows: 



seQ 



~H{{V)q\{V)i„^,X) VQCZm. (7) 

KCS 

-H{{Uhs^^\{V}i,^,X) ySCC (8) 

We follow the convention aw{4') = P{4>) = 0. Let i?^ V/C £ 
J^{C) and Ri yi E C be any set of rate tuples satisfying: 

J^R'k > awiQ)yQ^Iw,W EC 
iceQ 

les 

then, the RD region for the L— channel MD problem contains 
the rates and distortions for which there exist functions tps{-), 
such that 

Ri > R[+ J2 ^'^ 
JCeJii) 

Ds > E[ds{X,^Js{{V}j^s},{U}2S-^))] (10) 

The closure of the achievable tuples over all such 2^+^ — L — 2 
random variables is denoted by TZVcms- Observe that both 
the VKG and the CMS schemes are same as the ZB scheme 
for 2 descriptions scenario. 

III. Proof of strict improvement 

Note that, the total number of auxiliary random variables 
in the CMS scheme is almost twice that in the VKG scheme 
(which akeady is exponential in L). At the time of submission 
of |7|, it was yet unclear if this increase pays off with an 
improved achievable region. The following theorem, being the 
main contribution of this paper, establishes that there exists 
scenarios for which TZVcms is strictly larger than TZVvkg- 

Theorem 1. (i) The rate-distortion region achievable by 
the CMS scheme is always at least as large as the region 
achievable by the VKG region, i.e.: 
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VKG 



c 7^2?, 



CMS 



(11) 



( ii) There exists scenarios for which the CMS scheme leads 
to a region strictly larger than that achievable by the VKG 
scheme, i.e.: 



MS 



TIT) VKG c nv 



CMS 



(12) 



Specifically, for a binary symmetric source under Hamming 
distortion measure, the CMS scheme achieves a strictly larger 
rate-distortion region compared to the VKG scheme VL > 2. 

Proof Part (i) of the theorem is rather simple to prove 
and is a straight forward corollary of the main theorem in Q. 
It follows directly by setting Vs — ^ \/S such that \S\ < L in 
nVcMS- We then have aw{Q.) = VQ, < L. Substituting 
in (j9jl, we get > j3{S) + |5|aL('C) which is same as (j2ji. 



We prove (ii) by considering the binary symmetric source 
example for which the CMS scheme achieves points which 
cannot be achieved by the VKG scheme. Note that, once we 
prove that the CMS scheme achieves a strictly larger region 
for some L = I > 2, then it must be true for all L > I. Hence 
to prove (ii), it is sufficient for us to show that it is true for 
i = 3. However, we first include an example for L — A for 
building intuition and understanding of the type of scenarios 
where the CMS scheme provides strict improvement. Then we 
will prove the result for L = 3. We also note that obviously 
scenarios exit for which TZVcms = TIT^vkg (for example 
when TZVv KG is complete |3|). Finding the set of all such 
scenarios is an interesting problem in itself and is beyond 
the scope of this paper To describe our example, we require 
certain results pertaining to binary multiple descriptions |2| 
and successive refinement of binary sources ||9|, iflOl . In what 
follows, we state these results. 

The Zhang-Berger example : Zhang and Berger proved 
that, for the binary symmetric 2-descriptions MD prob- 
lem under Hamming distortion measure, sending a com- 
mon codeword in both the descriptions provides a strict 
improvement over the EC scheme. We briefly describe 
their result. Note that the rate-distortion region has 5 di- 
mensions denoted by {Ri, R2, Di, D2, Di2). Denote the 
rate-distortion region achievable by the EC scheme by 
Tl'DEc{bern{l/2)) and the corresponding region achievable 
by the ZB scheme (i.e. achieved by adding a common 
codeword among the two descriptions) by TZV zB{bern{l/2). 
Obviously ■RVEc{bern{l/2)) C nVzB{bern{\/2)), as we 
can always choose not to send any common codeword in the 
ZB scheme. Denote by Rec{D) the following cross section 
of 'RVEc{hern{l/2)): 

Rec{D) = inf|i?i + i?2 : Di + D2 < 2D 

iRi,R2,Di,D2,0) e nVEc{bern{l/2))} (13) 

Similarly, denote by Rzb{D), the corresponding cross section 
of nVzB{bern{l/2)). To show that nVEcibern{l/2)) C 
'R-'DzB{bern(l/2)), they considered a particular joint prob- 
ability mass function (PMF) P* {X,Vi2,Ui,U2,Ui2). Let 
us denote the achievable region associated with this PMF 
by TZT^ p* (bernjl / 2)) and the corresponding cross-section 
£3} by Rp,{D). They showed that 3D* > such that 
Rec{D*) > Rp*{D*) > RzBiD*). We refer the reader to 
llll for a detailed derivation and the values of P* and D* . 

Successive refinement : The problem of successive refine- 
ment was first proposed by Equitz and Cover in |9| and has 
since then been studied extensively by information theorists 
in. Ho]. The problem is motivated by scalable coding, where 
the encoder generates two layers of information called the base 
layer and the enhancement layer The base layer provides a 
coarse reconstruction of the source, while the enhancement 
layer is used to 'refine' the reconstruction beyond the base 
layer The objective is to encode the two layers such that 
the distortion at both the base and the enhancement layers 
are optimal. This setup is shown schematically in Figure 
3a Observe that, the 2-layer successive refinement region is 
indeed a special case (the cross-section {R1.R2, Di, 00, D12)) 



of the 2-descriptions MD setup where the distortion constraint 
on one of the individual descriptions is removed. The complete 
rate region for successive refinement was derived in 1 10] where 
it was shown that the EC coding scheme achieves the complete 
rate region. 

An interesting followup question is that of 'successive 
refinability' of sources. Assume di = d2 = d, then a source 
is said to be successively refinable under d if VI?i > D2, the 
rate point (i?i,i?2) = iTZVs{Di),nVsiD2) - nV,{Di)) is 
achievable, where TZ'Ds{D) denotes Shannon's rate distortion 
function. This condition implies that there is no loss in 
describing the source in two successive parts. An important 
point to note is that, for a successively refinable source, when 
the encoder operates at (i?i,i?2) = {T^T^s{D2)i'm^s{D2) — 
TZDsiDi)), there is absolutely no redundancy between the 
two layers of information, i.e., the two layers cannot carry a 
common codeword. We finally note that a binary symmetric 
source is successively refinable under Hamming measure |9|. 

Proof of (ii) : L = 4 : Consider a 4-descriptions MD 
problem for a binary symmetric source {hern{^)) under 
Hamming distortion measure. The rate-distortion region con- 
sists of 19 dimensions. We denote the region achievable 
by the VKG scheme by TZVyKO th^t achievable 

using the CMS scheme by TZV'^jf^.jg. We now consider 
a particular cross-section of these regions where we ap- 
ply constraints only on Di, D2, D12, D3 and Ds4. We re- 
move the constraints on all other distortions, i.e. we set 

D4, Di3, Di4, D23, D24D123, Di24, D134, D234 and L»i234 to 
00. Equivalently, we can think of a 4 descriptions MD problem 
with a particular channel failure pattern, wherein only one 
of the following sets of descriptions can reach the decoder 



reUably: (1, 2, 3, {1, 2}, {3, 4}) as shown in Figure |3b] We 
denote the set of all achievable points for this setup using 

the VKG and the CMS schemes by TZVyKO and TZV^jj^jg 
respectively. Note that, this equivalent model is used simply 
for analysis purposes, while we are actually interested in a 
cross section of the general binary symmetric 4-descriptions 
region. 

Observe that, with respect to the first 2 descriptions, we 
have a simple 2-descriptions problem and with respect to the 
last 2 descriptions, we have a successive refinement problem. 

Extending the arguments of Zhang and Berger, we define the 

4 

following infimum of TlVyj^Q : 



R 



VKG 



(D) = inf + i?2 : £>! + £'2 < 2D, 

4 

i?2, -R3, Ri, Di,D2,0, 03,034) e TZVyj^Q, 

R3 = nVs{D3), i?3 + i?4 = nVs{D34)] 



(14) 



Denote the corresponding infimum of TtVfjj^jg by RcMsi-^)- 
Recall that the VKG scheme forces all the descriptions to 
have a single common codeword. Constraints R3 ~ TZ'Ds{D3) 
and R3 + R4 ~ TZ'Ds{D34) ensure that descriptions 3 and 4 
carry completely complementary information, i.e. they cannot 
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Figure 3: (a)Successive refinement setup (b) Equivalent model for the considered cross-section of the 4-descriptions setup (c) 
Equivalent model for the considered cross-section of the 3 -descriptions setup 



carry a common codeword As VKG coding scheme forces 
the same common codeword among all the 4 descriptions, it 
follows that: 



Kkg{'D*) = Rec{D*) > Rzb{D*) 



(15) 



On the other hand, the CMS scheme allows for distinct 
common codewords to be sent in each subset of the de- 
scriptions. Hence, we can send a common codeword only 
among the two descriptions 1 and 2 while still maintaining 
i?3 = nVsiDs) and R^ + Ri^ nVs{D^i). This is achieved 
by setting all the common random variables to $ except V12, 
which has joint PMF P* with (X, J7i, C/2, C/12). This allows 
us to achieve: 



R%Ms{D*) = Rzb{D*) 



This impHes that RyKoi^*) > 



r4 

^CMS 



(16) 

(£>*) and hence 



TZVyj^Q C TZVfjj^jg. This example clearly illustrates the 
freedom the CMS scheme exhibits in controlling the redun- 
dancy across the messages. 

L = 3 : In similar lines to the 4-descriptions case, we 
next consider a 3 -descriptions MD problem for a binary 
symmetric source under Hamming distortion measure. Let 
the achievable regions be denote by TZVyj^Q and TZV^j^jg 
respectively. We consider the cross-sections of the achievable 
regions where we apply constraints only on Di, D2, D12 and 



£)i3 as shown in Figure 3c We denote these cross-sections 

^3 3 ' — ' 

by T^T^vKG '^^CMS respectively. Now consider any 
point {Ri, R2, R3, Di, D2, D12, D13) e TZV^KG such that 
{Ri,R2,Di,D2,Di2) e nVzB -TZVec and D13 < Di, 
where for two sets A and B, A-B = {I : I e A,l ^ B}. From 
the results of Zhang and Berger, if {Ri, R2, Di, D2, D12) S 
Tl'DzB — TIT) EC ^ descriptions 1 and 2 must carry a common 
codeword. Let the rate of the common codeword be R^ > 0. 
VKG scheme forces this codeword to be sent as part of R3 
as well. As this common codeword is received as part of both 

"^Note that, the constraint = 'R,'Ds{D'^) is in fact redundant. Just the 
constraint i?3 + i?4 = TZT>s{D-n) is sufficient to estabhsh that descriptions 
3 and 4 cannot carry a common codeword. However, as a binary source is 
successively refinable, we can always achieve D3 = H'DJ^ and hence 
the constraint i?3 = 'R,'Ds{D'^) gets applied implicitly once we apply + 
i?4 = 7?.I's(D34). This implies that, the gains due to the CMS scheme are 
not only restricted to successively refinable sources. In fact, the CMS scheme 
can achieve points outside the VKG region for any source and distortion 
measure for which the ZB scheme achieves points outside the EC scheme for 
the coiTesponding 2-descriptions setup. 



descriptions 1 and 3, it is redundant in R3 to achieve D13. This 
implies that i?2, i?3 -i?c, £'2, -D12, D13) e TZV^j^js- 
As there exit points in the boundary of TZVyp^Q which satisfy 
(Ri,R2,Di,D2,Di2) G nVzB - TI'Dec^ the CMS scheme 
achieves points outside the VKG scheme. Hence, we have 
shown that for a binary symmetric source under Hamming 
distortion measure, the CMS scheme achieves a strictly larger 
rate-distortion region than the VKG scheme for all L > 2. ■ 

IV. Conclusion 

We recently proposed a new encoding scheme for the 
general L— channel multiple descriptions problem involving 
'combinatorial message sharing' (CMS) which leads to a new 
achievable region subsuming the most well known region for 
this problem by Venkataramani, Kramer and Goyal (VKG) |l3l 
for general sources and distortion measures. In this paper, we 
showed that there exists scenarios (particularly for a binary 
symmetric source under Hamming distortion measure) for 
which, the new region is strictly larger than that achievable by 
the VKG scheme. As part of future work, we will investigate 
under what scenarios the CMS scheme achieves the complete 
RD region. 
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